1. Field
The present disclosure relates generally to application programs and, in particular, to testing application programs. Still more particularly, the present disclosure relates to a method and apparatus for testing application programs using application testing tools that are run on virtual machines.
2. Background
Many different types of complex systems are operated using application programs that require certification by one or more agencies. For example, without limitation, commercial aircraft, military aircraft, spacecraft, autonomous vehicles, nuclear power plants, medical equipment, and other types of complex systems require certification of different application programs. However, as application programs increase in size and complexity, it becomes increasing difficult to manually test these application programs and then verify that the results of this testing are accurate and complete.
Consequently, the testing and analysis of testing results are increasingly being performed through automated processes involving specialized computer programs, which may be referred to as application testing tools. But these application testing tools may themselves be a source of error that leads to testing results that are inaccurate, incomplete, or both.
Thus, these application testing tools may need to be verified and validated before they can be used to test application programs. For example, without limitation, before an application testing tool may be used on a particular hardware configuration to test an application program, the application testing tool may need to be qualified. Qualifying the application testing tool ensures that the application testing tool behaves as specified under all operational conditions. For example, an application testing tool may be qualified by verifying that the application testing tool meets all expected requirements when the application testing tool is run using a particular hardware and software configuration.
However, given the rate that computer hardware and software is evolving, it may be difficult to establish and maintain specific hardware configurations for running the application testing tools used to test application programs. In particular, requalifying an application testing tool after modifications, upgrades, or replacements have been made with respect to the original hardware and software configuration used to qualify the application testing tool, may be more difficult, time-consuming, and expensive than desired. Therefore, it would be desirable to have a method and apparatus that take into account at least some of the issues discussed above, as well as other possible issues.